Performance measurement tools assess the relative performance of an object based on an application running in a computing system. For example, the performance measurement tools apply benchmarks or tests to the computing system and measure the performance of computer hardware or the application. The performance measurement tool needs to be started and stopped at certain times so the tool can accurately measure the performance of the computer hardware or the application. For example, when an application starts running, the performance measurement tool may be subsequently started, and when the application stops performing the test, the performance measurement tool should be subsequently stopped. When operating in a single physical computing machine, a user may manually start and stop the performance measurement tool as needed. In this case, a user may determine that the application is starting and then start the performance measurement tool. The user may then determine when the application stops performing the test and stop the performance measurement tool. Because only a single physical computing machine is being used, the starting and stopping is not that much of a burden to the user.
In some cases, the tests are run “at scale”, which means multiple performance measurement tools are run concurrently. Having a user manually start and stop the performance measurement tool works well when a single test is being run in a single physical computing machine; however, when running at scale, manually starting and stopping multiple performance measurement tools becomes more difficult. For example, running at scale in a virtualization environment may require testing of hundreds of virtual machines running on a physical computing machine at once. In one example, each virtual machine includes a performance management tool that needs to be started and stopped, which makes the manual starting and stopping of the performance measurement tools very difficult. Further, the operation of virtual machines that are running on the physical computing machine may not be synchronized. For example, virtual machines being tested may not all start in the same instance and execute corresponding operations at the same time. Thus, each virtual machine may have a different start and stop time. A user easily cannot monitor each virtual machine and manually start and stop each performance measurement tool at multiple different times.